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DETAILED ACTION 
This is a Final rejection in response to Amendments/Remarks filed 05/21/2007. 
Claims 1,4-5, 8-13, 16-17, 20-25, 28-29, 32-36, and 44-52, are pending; 
Claims 2-3, 6-7, 14-15, 18-19, 26-27, 30-31, and 37-43 are previously canceled. 
Claims 1,13, and 25 are independent claims. 
Effective filing date is 3-15-2000 (Ricoh). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

Claims 1, 4-5, 8-11, 13, 16-17, 20-23, 25, 28-29, 32-35, and 44-52 are rejected 
under 35 U.S.C. 103(a) as being unpatentable over Schena et al. [hereinafter 
Schena], US 6,448,979 B1 filed 1/25/1999, in view of Robinson etal. "The Origami 
Project: Paper Interfaces to the World-Wide Web", 

[http://www.cl.cam.ac.uk/Research/Origami/Origami1997f/index.html], submitted 
to Webnet 97 in November 1997 [hereinafter Robinson], further in view of Gornish 
et at. US patent 5,337,362 issued 08/09/1994 [hereinafter Gornish]. 
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Regarding independent claim 1: 

Schena teaches the multimedia annotation representing at least one of an 
audio sound and a video clip, wherein the at least one of the audio sound and 
video clip to be extracted from the multimedia annotation, and the at least one 
extracted audio sound and video clip can be played via a multimedia player, and 
the multimedia annotation represented by a first bar code encoding at least one 
of an audio sound and a video clip, wherein the at least one of the audio sound 
and video clip. Specifically, Schena discloses the multimedia annotation such that the 
multimedia annotation can be extracted and decoded subsequently from the first 
multimedia document, wherein the scanner detects and reads data, such as machine- 
readable codes containing link information corresponding to provider information from 
the printed medium. The link information corresponding to the provider information may 
include, a universal resource locator (URL), such as customer premises equipment 
(CPE) for displaying the multimedia sequence information. The multimedia sequence 
information may be advertising or transaction information and may contain one or more 
of textual, audio, or video information, and the customer premises equipment ("CPE"), 
serving as the receiver, plays multimedia sequence (Schena col.1, line 64 through col. 
2. line 40). In addition, Schena discloses the scanner is capable of reading data such as 
non-coded data and machine-readable code 10 from the printed medium 50. The 
machine-readable code 10 (URL) may be a barcode, an enhanced barcode, a new 
enhanced code, or any type of code, including dynamic codes and high-density 
barcodes (Schena col. 3, lines 55-65). 
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The Examiner equates the claimed the multimedia annotation representing at 
least one of an audio sound and a video clip as equivalent to the link information 
may contain one or more of textual, audio, or video information as taught by Schena, 
and the claimed a multimedia as equivalent to the plays multimedia sequence as 
taught by Schena. Also, the Examiner equates the claimed bar code encoding at least 
one of an audio sound and a video clip as equivalent to the link information from 
barcode, code 10 (URL) may contain one or more of textual, audio, or video information 
as taught by Schena, and because the Applicant invention specification discloses "The 
annotation may be in different forms, such as, for example, a bar code containing an 
audio message or a URL indicating a link to a video clip. " (Applicants invention 
specification page 7 lines 15-18). 

In addition, Schena does not explicitly teach, but Robinson teaches creating a 
multimedia annotation for a paper document, and creating a first multimedia 
document by combining the paper document and the multimedia annotation, 
wherein the first multimedia document is generated as a part of reproducing the 
paper document via a document reproduction system. Specifically, Robinson 
discloses the origami project, wherein the printed documents are annotated with marks 
in their corners to facilitate recognition and location on the desk top, and are also have a 
unique identifier printed in an OCR fount (see Robinson page 4 section Printing), these 
steps involve recognizing that a page printed by the system has appeared on the desk, 
determining its position, reading its unique identifier and locating any interactors. A 
transformation is then set up between the page representation stored in the registry and 
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physical co-ordinates on the desktop. The printed document thus becomes part of the 
projected window system, wherein any active links are highlighted by projecting a red 
background over them. For a document originating on the Web, these correspond to 
links in the original HTML (Robinson page 4 section DigitalDesk, also see fig. 2 a-c, as 
example of the process of annotating a printed media). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Schena to include a method for creating a multimedia annotation for 
a paper document, and creating a first multimedia document by combining the paper 
document and the multimedia annotation, wherein the first multimedia document is 
generated as a part of reproducing the paper document via a document reproduction 
system, as taught by Robinson. One of ordinary skill in the art would have been 
motivated to modify this combination, because Schena and Robinson are from the 
same field of endeavor of providing multimedia annotation for a paper document and for 
the advantage of bridging the gap between the virtual multimedia-based of the Internet 
word and the physical world of printed media, wherein the annotate media may contain 
one or more of textual, audio, or video information, and the customer premises 
equipment ("CPE"), serving as the receiver, plays multimedia sequence (Schena col.1, 
lines 5-15, col. 1 line 64 through col. 2. line 40). 
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University of Cambridge Computer Lab 



Thi> is the home page at the University of Cambridge Computer Laboratory , Our official World Wfl& Web 
server Is www cl. ran.ac.i*. 

We are the Computer Science Teaching and Research department You need to look els ewhere fcr information 
about tht imfrarrf-y aiawhole. or about the llnivpjsitv rumouring Service. 



Local information 

• An overview of the Computer Laboratory 

• Ust oi menben of the Department, alsa available for TABIE capable browsers 

• ah Kn?wn information s erven fa CambridRe 

• Cambridge Unfrersfcy Information System 

• ATI newsgroups avaflahlf.in trt». local rmws sprvrr 

• Caippu^r nqwygrpupsj 

o u&acadjnisc 
<j ucam,cl,itudents 

• Anonymous FT P area ford.cam.ac.uk ("also available via WW^ 

• Maps of th» Computer Lab and How to get here 

• Rnd sameone using their Active Badge. There's a Graphical Overview of tht Active Badge system. 
(Some of this information is only available to local mzcnines - sorry) 

• Ubrwyinfyrmgtion 

• The status of the Trojan 3oom Coffee Machine 

• Usage statistics for this s=rvcr, updated dailv. 

• Information and recruitment details for members of the Ccmputer Laboratory's industrial 
•Supponcn-Cfab' 

• Job vacancies in the Computer Laboratory 

• Annowc^entif of Computer laboratory Seminars 



For users in cl.cam.ac.uk only: 

• Local system information page 

• How to export personal ties to the WWW 



fa) Original Web page 
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ucam.cl .st udehts 
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Local system information page 
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(c) Animated on the DigitalDesk. 
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In addition, Schena, and Robinson do not expressly teach, but Gornish teaches: 

wherein the multimedia annotation is captured via an input device of 
the document reproduction system while the paper document is being 
reproduced via the document reproduction system, wherein the captured 
multimedia annotation is encoded within the first bar code, 
(See Gornish at Column 2, Lines 5-25, teaching a method and apparatus for 
transferring digital information (each box includes a frame or border. Binary data is 
formatted in rows within the box, wherein a bit of digital data is depicted by the presence 
or absence of an ink dot. The inside edges of the left and right sides of the box have 
markers for identifying the vertical separations (i.e., the rows) between the digital data) 
to and from (encode and decode) plain paper is described, a reproduction system 
having an encryption device for performing encryption on a data source to produce 
encrypted data. In addition, also includes an encoder for encoding the encrypted data 
into a series of pixel values. These pixel values are then output by a printing device, 
such that the data is placed in multiple boxes having a frame and a plurality of rows and 
columns on a piece of plain paper, such that the data source is represented in 
encrypted digitized form on the plain paper, 

wherein the first multimedia document, which when scanned by a 
process, the process causes the printed multimedia annotation to be 
decoded. 
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(See Gornish at Column 2. Lines 30-40, teaching recognizing the data on the plain 
paper, for recognizing the data includes a scanning device for scanning the boxes 
(barcode) on the plain paper and for converting the pixels into electrical signals 
representative of the characters. And, also includes a decoder for decoding the 
electrical signals into output signals representative of the data. A decryption device then 
performs decryption on the decoded data. An output device then transfers the decrypted 
data onto a piece of plain paper. It is noted the claimed barcode is digital information 
(each box includes a frame or border. Binary data is formatted in rows within the box, 
wherein a bit of digital data is depicted by the presence or absence of an ink dot. The 
inside edges of the left and right sides of the box have markers for identifying the 
vertical separations (i.e., the rows) between the digital data as taught by Gornish.) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Schena and Robinson, to include a means wherein the multimedia 
annotation is captured via an input device of the document reproduction system while 
the paper document is being reproduced via the document reproduction system, 
wherein the captured multimedia annotation is encoded within the first bar code, the first 
multimedia document, which when scanned by a process, the process causes the 
printed multimedia annotation to be decoded as taught by Gornish. One of ordinary skill 
would be motivated to combine a method for transferring digital data (i.e. barcode) to 
and from (encode and decode) plain paper of Jarman to encode and decode the URL 
indicating a link to a video/audio clip of Schena and DigitalDesk of Robinson to 
providing multimedia annotation for a paper document and for the advantage of bridging 
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the gap between the virtual multimedia-based of the Internet word and the physical 
world of printed media, wherein the annotate media may contain one or more of textual, 
audio, or video information, and the customer premises equipment ("CPE"), serving as 
the receiver, plays multimedia sequence (Schena col.1, lines 5-15, col. 1 line 64 
through col. 2. line 40). 

Regarding independent claim 13, 

Is directed to machine-readable medium providing instructions, which 
when executed by a set of one or more processors to perform the method of 
claim 1 which cites above, and is similarly rejected under the same rationale, 
Also See Schena Fig, 1. 

Regarding independent claim 25, 

Is directed to a computer system include a bus, a data storage, and a 
processor to perform the method of claim 1 which cites above, and is similarly 
rejected under the same rationale, Also See Schena Fig, 1. also Column 2, lines 
5-20). 

Regarding claims 4, 16, and 28, 

Schena teaches wherein a location indicator associated with the multimedia 
annotation is placed on the first multimedia document, wherein the location 
indicator indicates where the multimedia annotation can be retrieved and played. 
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Specifically, Schena discloses the link information corresponding to the provider 
information may include, a universal resource locator (URL), such as customer 
premises equipment (CPE) for displaying the multimedia sequence information. The 
multimedia sequence information may be advertising or transaction information and 
may contain one or more of textual, audio, or video information, and the customer 
premises equipment ("CPE"), serving as the receiver, plays multimedia sequence 
(Schena col.1, line 64 through col. 2. line 40). 

Regarding claims 5, 17, and 29, 

Schena teaches wherein the location indicator comprises a first Uniform 
Resource Locator (URL), and a second bar code, wherein the first URL is 
indicated in plain text, and wherein the second bar code represents the first URL 
in an encrypted form. Specifically, Schena discloses the link information 
corresponding to the provider information may include, a universal resource locator 
(URL) (Schena col.1, line 64 through col. 2. line 40). In addition, the links can be 
encoded according to provider, for example UPC or ISBN numbers and any code may 
serve as the alphanumeric sequence (Schena col. 4, lines 45-55). 

The Examiner equates the claimed bar code as equivalent to the UPC or ISBN 
numbers and any code as taught by Schena, and because the Applicant invention 
specification discloses "The annotation may be in different forms, such as, for example, 
a bar code containing an audio message or a URL indicating a link to a video clip. " 
(Applicants invention specification page 7 lines 15-18). 
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Regarding claims 8, 20, and 32, 

the rejection of claim 1 is fully incorporated. In addition, 

Schena does not explicitly teach, but Robinson teaches, generating an image 
of the paper document, the image of the paper document being unconsciously 
captured via the document reproduction system during the reproduction of the 
paper document without user intervention. Specifically, Robinson discloses the 
animated paper involves scanning image of the page from top to bottom, left to right and 
emitting text or images as appropriate. One of these is for the HTML of the page itself 
(Robinson page 4 the Export section). 

The Examiner equates the claimed unconsciously and without user 
intervention as equivalent to the animated paper involves scanning image of the page 
itself as taught by Robinson. 

Also, Robison teaches storing the image of the paper document and the 
multimedia annotation in a storage, wherein the second multimedia document is 
an electronic document associated with the first multimedia document, which is a 
physical document Specifically, Robinson discloses a Registry, which maintains the 
association between electronic documents and their printed variants. It stores the image 
of each active document and the code of any interactions required for the document, 
together with cross references between these and further indexes to identify them 
(Robison page 3 the Registry section, also see Fig. 1). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Schena to include a method for generating an image of the paper document, 
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the image of the paper document being unconsciously captured via the document 
reproduction system during the reproduction of the paper document without user 
intervention, and storing the image of the paper document and the multimedia 
annotation in a storage, wherein the second multimedia document is an electronic 
document associated with the first multimedia document, which is a physical document 
as taught by Robinson. One of ordinary skill in the art would have been motivated to 
modify this combination, because Schena and Robinson are from the same field of 
endeavor of providing multimedia annotation for a paper document and for the 
advantage of bridging the gap between the virtual multimedia-based of the Internet word 
and the physical world of printed media (Schena col.1, lines 5-15, col. 1 line 64 through 
col. 2. line 40), and allowing the database to be built and edited, imported and exported 
to other forms of hypertext, and for documents to be printed and animated on a 
DigitalDesk (Robison page 32 the Registry section). 




Oesk surface 



Figure 1: Animated paper document framework. 
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Regarding claims 9, 21, and 33, 

Schena teaches wherein a multimedia document is represented as a second 
Uniform Resource Locator (URL) printed on the first multimedia document, and 
wherein the image of the paper document and the multimedia annotation is 
accessed with the second URL (Schena col. 1 line 61 - col. 2 line 25 and col. 3 line 
64 - col. 4 line 6). 

Regarding claims 10, 22, and 34, 

Schena teaches wherein a third bar code is used to represent a second 

URL (Schena in col. 3 lines 58-63). 

Regarding claims 11, 23, and 35, 

Schena teaches automatically sending a second multimedia document to a 
recipient by electronic mail as a part of reproducing the paper document via the 
document reproduction system, wherein the recipient is specified by a user via 
an interface of the document reproduction system when the user reproduces the 
paper document using the document reproduction system. Specifically, Schena 
discloses the link information was published or located, along with message-specific 
information (Schena col. 4 line 35). 

The examiner equates the claimed automatically sending a second 
multimedia document to a recipient by electronic mail as equivalent to link 
information was published or located, along with a message as taught by Schena. 
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Regarding claim 44, 

Schena teaches wherein the first multimedia document is a physical 
document having the first bar code printed thereon, which when scanned by a 
scanning device, causes the first bar code to be decoded and the audio sound to 
be extracted from the first bar code, and wherein the extracted audio sound is 
capable of being played via an audio player. Specifically, Schena discloses the 
multimedia annotation such that the multimedia annotation can be extracted and 
decoded subsequently from the first multimedia document, wherein the scanner detects 
and reads data, such as machine-readable codes containing link information 
corresponding to provider information from the printed medium. The link information 
corresponding to the provider information may include, a universal resource locator 
(URL), such as customer premises equipment (CPE) for displaying the multimedia 
sequence information. The multimedia sequence information may be advertising or 
transaction information and may contain one or more of textual, audio, or video 
information, and the customer premises equipment ("CPE"), serving as the receiver, 
plays multimedia sequence (Schena col.1, line 64 through col. 2. line 40). In addition, 
Schena discloses the scanner is capable of reading data such as non-coded data and 
machine-readable code 10 from the printed medium 50. The machine-readable code 10 
(URL) may be a barcode, an enhanced barcode, a new enhanced code, or any type of 
code, including dynamic codes and high-density barcodes (Schena col. 3, lines 55-65). 

The Examiner equates the claimed bar code as equivalent to the link information 
from barcode, code 10 (URL) may contain one or more of textual, audio, or video 



Application/Control Number: 09/526,031 Page 17 

Art Unit: 2176 

information as taught by Schena, and because the Applicant invention specification 
discloses "The annotation may be in different forms, such as, for example, a bar code 
containing an audio message or a URL indicating a link to a video clip. " (Applicants 
invention specification page 7 lines 15-18). 

Regarding claim 45, 

Schena teaches capturing an audio sound of the multimedia annotation 
from a user using a microphone of the input device to annotate the paper 
document to create a multimedia paper document. Specifically, Schena discloses an 
enhanced electronic device, digital appliances, and microphone (Schena col. 3, lines 
40-65). Also, Schena discloses the user interface obtains user input information. The 
user interface may be, for example, a voice-activated system, a keypad, or a keyboard 
(Schena col. 2, lines 25-45). 

Regarding claim 46, 

Schena teaches wherein the microphone is automatically activated when 
the user selects a reproduction function of the document reproduction system to 
reproduce the paper document. Specifically, Schena discloses an enhanced 
electronic device, digital appliances, and microphone (Schena col. 3, lines 40-65). Also, 
Schena discloses the user interface obtains user input information. The user interface 
may be, for example, a voice-activated system, a keypad, or a keyboard (Schena col. 2, 
lines 25-45). 
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The examiner equate the microphone is automatically activated as equivalent 
to a voice-activated system as taught by Schena. 



Regarding dependent claim 47, 

Schena teaches capturing a video clip of the multimedia annotation from a 
user using a video camera of the input device to annotate the paper document to 
create a multimedia paper document. Specifically, Schena discloses an enhanced 
electronic device, digital appliances, and microphone (Schena col. 3, lines 40-65). Also, 
Schena discloses the user interface obtains user input information. The user interface 
may be, for example, a voice-activated system, a keypad, or a keyboard (Schena col. 2, 
lines 25-45). 

The examiner reads a video camera of the input device as equivalent to digital 
appliances, and microphone as a user interface for obtaining user input information as 
taught by Schena. 

Regarding dependent claim 48, 

Schena teaches wherein the video camera is automatically activated when 
the user selects a reproduction function of the document reproduction system to 
reproduce the paper document. Specifically, Schena discloses an enhanced 
electronic device, digital appliances, and microphone (Schena col. 3, lines 40-65). Also, 
Schena discloses the user interface obtains user input information. The user interface 
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may be, for example, a voice-activated system, a keypad, or a keyboard (Schena col. 2, 
lines 25-45). 

The examiner equates the video camera is automatically activated as 

equivalent to digital appliances, and microphone as a user interface for obtaining user 
input information as taught by Schena. 

Regarding dependent claim 49, 

Schena does not teach in response to a request to retrieve a second 
multimedia document, performing a content-based search for the requested 
multimedia document within the storage based on the content of the multimedia 
annotation associated with the requested multimedia document Robinson does 
teach parsing and indexing the multimedia documents contained in the registry for 
retrieval in fig. 1 and (the "Registry" section in pages 2-3). It would have been obvious to 
one of ordinary skill in the art at the time the invention was made to have combined 
Robinson into Schena to have created the claimed invention. It would have been 
obvious and desirable to have used the registry index of Robinson to have searched 
and located the appropriate multimedia document to retrieve. 

Regarding dependent claims 50-52, 

Schena does not teach wherein the content-based search is performed by 
OCR, audio speech recognition, or video face recognition techniques on the 
multimedia annotations of the multimedia documents being searched, Robinson 
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does teach parsing and indexing the contents of the multimedia documents contained in 
the registry using OCR, audio speech recognition, or video face recognition techniques 
(in fig. 1 and the "Registry" section in pages 2-3). It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to have combined Robinson 
into Schena to have created the claimed invention. It would have been obvious and 
desirable to have used the registry index of Robinson to have searched and located the 
appropriate multimedia document to retrieve. 

Claims 12, 24, and 36 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Schena et al. (hereinafter Schena), US 6,448,979 B1 filed 
1/25/1999, in view of Robinson et al. "The Origami Project: Paper Interfaces to the 
World-Wide Web", 

[http://www.cI.cam.ac.uk/Research/Origami/Origami1997f/index.html], submitted 
to Webnet 97 in November 1997 (hereinafter Robinson), further in view of Gornish 
et at. US patent 5,337,362 issued 08/09/1994 [hereinafter Gornish], and further in 
view of Hallida v et al., US 5,880,740 filed 7/12/1996 (hereinafter Halliday). 

Regarding claims 12, 24, and 36, 

Schena, Robinson, and Gornish do not teach, but Halliday teaches wherein the 
recipient receives the image of the paper document and the multimedia 
annotation in the form of Multi-purpose Internet Mail Extension (MIME). Halliday 
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does teach sending an image of a document in the form of Multi-purpose Internet 
Mail Extension (MIME). Specifically, Halliday discloses MIME (Halliday col. 8, lines 5- 
28). 

It would have been obvious and desirable to have used the Multi-purpose 
Internet Mail Extension (MIME) teaching of Halliday to have implemented the automatic 
sending of the electronic multimedia document of Schena in view of Robinson, and 
further in view of Jarman so that the recipient would have used any common email 
client to have received the electronic multimedia document sent from the user. 



It is noted that any citations to specific, pages, columns, lines, or figures in the 
prior art references and any interpretation of the references should not be considered to 
be limiting in any way. A reference is relevant for all it contains and may be relied upon 
for all that it would have reasonably suggested to one having ordinary skill in the art. 
See, MPEP2123. 
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Response to Arguments 

Brief description of cited prior art: 

Schena [hereinafter Schena] discloses the machine-readable code on the paper is used 
to communicate corresponding multimedia information when the machine-readable 
code is read by a scanner. The URL encoded in the machine-readable code points to a 
multimedia file. Thus, the machine-readable code on the printed medium annotates the 
printed medium with the referenced multimedia file, thus combining the printed medium 
with virtual multimedia. In combining the multimedia presentation with the printed 
medium, Schena has created a multimedia document of the claimed invention (See 
Schena the Abstract) and also Schena in col. 2 lines 6-8 that the multimedia annotation 
may contain one or more of textual, audio, or video information. 

Robinson [hereinafter Robinson] discloses the origami project, wherein the printed 
documents are annotated with marks in their corners to facilitate recognition and 
location on the desk top, and are also have a unique identifier printed in an OCR fount 
(see Robinson page 4 section Printing), these steps involve recognizing that a page 
printed by the system has appeared on the desk, determining its position, reading its 
unique identifier and locating any interactors. A transformation is then set up between 
the page representation stored in the registry and physical co-ordinates on the desktop. 
The printed document thus becomes part of the projected window system, wherein any 
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active links are highlighted by projecting a red background over them. For a document 
originating on the Web, these correspond to links in the original HTML (Robinson page 
4 section DigitalDesk, also see fig. 2 a-c, as example of the process of annotating a 
printed media). 

Gornish [hereinafter Gornish] discloses a method and apparatus for transferring digital 
information (each box includes a frame or border. Binary data is formatted in rows within 
the box, wherein a bit of digital data is depicted by the presence or absence of an ink 
dot. The inside edges of the left and right sides of the box have markers for identifying 
the vertical separations (i.e., the rows) between the digital data) to and from (encode 
and decode) plain paper is described. 

Halliday [hereinafter Halliday] discloses MIME (Halliday col. 8, lines 5-28). 

Beginning on page 13 of the Remarks (hereinafter the remarks), Applicant 
argues the following issues, which are accordingly addressed below. 

Rejection of Claims 1, 4-5, 8-11, 13, 16-17, 20-23, 25, 28-29, 32-35, and 44-52 
Under 35 U.S.C. § 103(a) over Schena and Robinson: 
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Firstly: Applicant argues, the combination of Schena and Robinson fail to teach the 
features of " the machine-readable code of the printed medium represents at least one 
of an audio clip and a video clip via a bar code that encodes the audio and/or video 
clips/' because Schena do not teach or suggest: "encode" at least the one of an audio 
sound and a video clip, See the remarks Page 14 Last Para -> Page 15 First Para). 

The examiner disagrees. 

For purposes of responding to Applicant's argument, the examiner will assume that Applicant is 
arguing for the patentability of Claim 1. 

As discuss in the rejection above, specifically Schena discloses, the machine- 
readable code on the paper is used to communicate corresponding multimedia 
information when the machine-readable code is read by a scanner The URL encoded 
in the machine-readable code points to a multimedia file. Thus, the machine-readable 
code on the printed medium annotates the printed medium with the referenced 
multimedia file, thus combining the printed medium with virtual multimedia. In 
combining the multimedia presentation with the printed medium, Schena has created a 
multimedia document of the claimed invention (See Schena the Abstract) and also 
Schena in col. 2 lines 6-8 that the multimedia annotation may contain one or more of 
textual, audio, or video information. 

In addition, Gornish discloses a method and apparatus for transferring digital 
information (each box includes a frame or border. Binary data is formatted in rows within 
the box, wherein a bit of digital data is depicted by the presence or absence of an ink 
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dot. The inside edges of the left and right sides of the box have markers for identifying 
the vertical separations (i.e., the rows) between the digital data) to and from (encode 
and decode) plain paper is described. It is noted the claimed barcode is digital 
information (each box includes a frame or border. Binary data is formatted in rows within 
the box, wherein a bit of digital data is depicted by the presence or absence of an ink 
dot. The inside edges of the left and right sides of the box have markers for identifying 
the vertical separations (i.e., the rows) between the digital data as taught by Gornish.) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Schena, to include a means of encoding the barcode of Gornish, 
One of ordinary skill would be motivated to combine a method of transferring digital 
information to and from (encode and decode) plain paper of Gornish to encoded URL in 
the machine-readable code points to a multimedia file. Thus, the machine-readable 
code on the printed medium annotates the printed medium with the referenced 
multimedia file, thus combining the printed medium with virtual multimedia. In 
combining the multimedia presentation with the printed medium, Schena has created a 
multimedia document of the claimed invention (See Schena the Abstract) and also 
Schena in col. 2 lines 6-8 that the multimedia annotation may contain one or more of 
textual, audio, or video information. 

Therefore, Schena and Gornish clearly teach, "the machine-readable code of the 
printed medium represents at least one of an audio clip and a video clip via a bar code 
that encodes the audio and/or video clips". Thus, Robinson need not teach these 
limitations to support a proper 103 rejection. 
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Secondly: Applicant argues, "the multimedia annotation may be a bar code 
"containing" an audio message, which is clearly not in a form of linkage , and the 
claimed language clearly indicates such a limitation." See the remarks Page 15, 
Second). 

The examiner disagrees. 

For purposes of responding to Applicant's argument, the examiner will assume that Applicant is 
arguing for the patentability of Claim 1. 

As discuss in the rejection above, Firstly, the Applicant invention specification 
discloses, "The annotation may be in different forms, such as, for example, a bar code 
containing an audio message or a URL indicating a link to a video clip. " (Applicants 
invention specification page 7 lines 15-18). Where the Examiner interpreted the claimed 
in light of the specification, thus it would have been obvious to one of ordinary skill in the 
art to substitute applicant's barcode with applicant's URL indicating a link to a video clip 
to provide access to a video clip. Secondly, Applicant admitted, "the multimedia 
annotation may be a bar code", the phase "may be" rendering the uncertainty of choice, 
which is not limited only to the barcode and not to a URL indicating a link to a video clip. 

To clarify, the Examiner introduce Gornish reference as discuss in the rejection 
above, specifically, Gornish discloses a method and apparatus for transferring digital 
information (each box includes a frame or border. Binary data is formatted in rows within 
the box, wherein a bit of digital data is depicted by the presence or absence of an ink 
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dot. The inside edges of the left and right sides of the box have markers for identifying 
the vertical separations (i.e., the rows) between the digital data) to and from (encode 
and decode) plain paper is described. It is noted the claimed barcode is digital 
information (each box includes a frame or border. Binary data is formatted in rows within 
the box, wherein a bit of digital data is depicted by the presence or absence of an ink 
dot. The inside edges of the left and right sides of the box have markers for identifying 
the vertical separations (i.e., the rows) between the digital data as taught by Gornish.) 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify Schena, to include a means of encoding the barcode of Gornish. 
One of ordinary skill would be motivated to combine a method of transferring digital 
information to and from (encode and decode) plain paper of Gornish to encoded URL in 
the machine-readable code points to a multimedia file. Thus, the machine-readable 
code on the printed medium annotates the printed medium with the referenced 
multimedia file, thus combining the printed medium with virtual multimedia. In 
combining the multimedia presentation with the printed medium, Schena has created a 
multimedia document of the claimed invention (See Schena the Abstract) and also 
Schena in col. 2 lines 6-8 that the multimedia annotation may contain one or more of 
textual, audio, or video information. 
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Thirdly: Applicant argues Schena and Robinson fail to teach, "the multimedia 
annotation is captured via an input device of the document reproduction system while 
the paper document is being reproduced via the document reproduction system, where 
the captured multimedia annotation is encoded within the first bar code." See the 
remarks Page 15, Third Para). 

The examiner disagrees. 

For purposes of responding to Applicant's argument, the examiner will assume that Applicant is 
arguing for the patentability of Claim 1 . 

As discuss in the rejection above, to address the newwly amended portion, the 
Examiner introduces the Gornish reference (See the rejection above for dettails). 

Fourthly: Appellant argues, the combination of Schena, Robinson, and Halifax fail to 
render the obviousness under 35 U.S.C. 103 (a), See the remarks Page 15 Last 
Paragraph -> Page 17. 

The examiner disagrees. 

Following KSR direction as following: "SUPREME COURT OF THE UNITED 
STATES No. 04-1350 KSR INTERNATIONAL CO., PETITIONER v. TELEFLEX INC. 
ET AL. ON WRIT OF CERTIORARI TO THE UNITED STATES COURT OF APPEALS FOR 
THE FEDERAL CIRCUIT [April 30, 2007], (page 2-3 of the court opinion) Following Graham v. 
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John Deere Co. of Kansas City, 383 U. S. 1 (1966), the Court set out a framework for applying 
the statutory language of §103, language itself based on the logic of the earlier decision in 
Hotchkiss v. Greenwood, 11 How. 248 (1851), and its progeny. See 383 U. S., at 15-17. The 
analysis is objective: 

"Under §103, the scope and content of the prior art are to be determined; differences 
between the prior art and the claims at issue are to be ascertained; and the level of ordinary 
skill in the pertinent art resolved. Against this background the obviousness or 
nonobviousness of the subject matter is determined. Such secondary considerations as 
commercial success, long felt but unsolved needs, failure of others, etc., might be utilized to 
give light to the circumstances surrounding the origin of the subject matter sought to be 
patented." Id., at 17-18. 

While the sequence of these questions might be reordered in any particular case, the factors 
continue to define the inquiry that controls. If a court, or patent examiner, conducts this analysis 
and concludes the claimed subject matter was obvious, the claim is invalid under §103. Seeking 
to resolve the question of obviousness with more uniformity and consistency, the Court of 
Appeals for the Federal Circuit has employed an approach referred to by the parties as the 
"teaching, suggestion, or motivation" test (TSM test), under which a patent claim is only proved 
obvious if "some motivation or suggestion to combine the prior art teachings" can be found in 
the prior art, the nature of the problem, or the knowledge of a person having ordinary skill in the 
art See, e.g., Al-Site Corp. v. VSI Int'l, Inc., 174 F. 3d 1308, 1323-1324 (CA Fed. 1999). KSR 
challenges that test, or at least its application in this case. See 119 Fed. Appx. 282, 286-290 
(CA Fed. 2005). Because the Court of Appeals addressed the question of obviousness in a 

manner contrary to §103 and our precedents, we granted certiorari, 547 U. S (2006). We 

now reverse. 

Using the broadest reasonable interpretation, and cites evidences above, the 
Examiner had found that Schena in view of Robinson, further in view of Halliday, and 
newly added Gornish reference have taught all the limitation of the claimed invention. 

In addition, As discuss in the section First^third Response to Argument cites 
above, thus the examiner has established "some motivation or suggestion to combine the 
prior art teachings" can be found in the prior art, the nature of the problem, or the knowledge of 



a person having ordinary skill in the art. See, e.g., Al-Site Corp. v. VSI Int'l, Inc., 174 F. 3d 1308, 



Application/Control Number: 09/526,031 Page 30 

Art Unit: 2176 

1323-1324 (CA Fed. 1999). KSR challenges that test, or at least its application in this case. See 
119 Fed. Appx. 282, 286-290 (CA Fed. 2005). 

Accordingly, for at least all the above evidence, and the current rejection, 
therefore the Examiner respectfully maintains the rejection of claims 1, 4-5, 8-13, 16-17, 
20-25, 28-29, 32-36, and 44-52, at least at this time. 



Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .1 36(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Quoc A. Tran whose telephone number is 571-272- 
8664. The examiner can normally be reached on Monday through Friday from 9 AM to 
5 PM EST. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Doug Hutton can be reached on 571-272-4137. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

Quoc A, Tran 
Patent Examiner 
Art Unit 2176 
08/04/2007 
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